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ABSTRACT 

This paper discusses some of the problems encountered 
in the design of discrete-time stochastic controllers for 
problems that may adequately be described by the "LQG" 
assumptions; namely, the problems of obtaining acceptable 
relative stability, robustness, and disturbance rejection 
properties. The paper proposes a dynamic compensator to 
replace the optimal full state feedback regulator gains at 
steady state, provided that all states are measurable. The 
compensator increases the stability margins at the plant 
input, which may possibly be Inadequate in practical 

applications. Though the optimal regulator has desirable 

properties the observer based controller as Implemented with 
a Kalman filter. In a noisy environment, has inadequate 
stability margins. The proposed compensator is designed to 
match the return difference matrix at the plant input to that 
of the optimal regulator while maintaining the optimality of 
the state estimates as dictated by the measurement noise 
character i sties. 
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I. INTRODUCTION 


The design of robust stochastic controllers for 
problems adequately described by the "LQG" assumptions has 
been a field of active research in recent years. Since 
Doyle's [1] and Doyle and Stein's [2] Introductory papers 
d ? f f erent approaches have been taken i n order to des i gn 
various robust LQG controllers. It can be stated generally 
that the approaches taken were to increase the stability 
margins, namely the gain and phase margins at the plant 
input, sufficiently so that the closed loop system remained 
stable under large parameter changes in the plant and/or 
sensor failures. It is Important to note at this point that 
most research has been on continuous time systems. The 
robustness problem may be more pronounced in discrete time 
controllers due to sampling rate limitations and the phase 
lag associated with sampl Ing. 

In order to have a better understanding of the problem 
it is necessary to briefly review the respective parts of the 
stochastic controller. The stochastic LQG controller is 
comprised of the LQ optimal feedback controller and the 
Kalman filter based current full state observer. It has been 
well established that the continuous time LQ controller based 
system has excellent guaranteed stability margins, namely a 
phase margin of at least 60° and an infinite gain margin. 
Unfortunately the discrete time equivalent doesn't have these 
guaranteed margins. However as the sampling period 
approaches zero the stability margins approach those that of 
the continuous time LQ controller. The Kalman filter will 



show an excellent performance In estimating states and will 
also be stable. However when the Kalman filter Is used to 
estimate the state variables for feedback to the LQ 
controller the robustness properties of the system will not 
be guaranteed. Doyle has given a simple example where a LQG 
controller-filter combination has very small gain margins, 
and hence Is not robust. An Investigation of the paper by 
Johnson [3] explains this behavior of the LQG controllers. 

Consider the state space representation of a plant for 
which a LQG control 1 er i s to be designed. 

x(k+l ) = Ax(k) + Bu(k) + Gw(k) 


y(k) = Cx(k) + v(k) 


( 1 ) 


where 

x(k)eR n , u(k)eR r , y(k)eR m 

and w(k) and v(k) are uncorrelated, zero mean white gauss Ian 
noise processes. 

Denoting the constant Kalman filter gains by K^. and the 
constant LQ gains by K c we consider the discrete time 
equivalent of Theorem 8.3 as stated in the monograph by 
0' Re 1 1 ly [4] . 

Theorem : There exists a class of linear systems ( 1 ) such 

that one or more eigenvalues of (1 - K^.C) (A - BK c > of the 

observer based feedback controller may lie outside of the 

unit circle in the complex plane though all eigenvalues of (A 

- BK ) and all the eigenvalues of (A - K,CA) are designed to 
C r 

lie within the unit circle, and even though the system pairs 
( A , B ) , ( A , C ) are, respectively completely controllable and 

completely observable. 


The significance of this theorem lies in the fact that 
although the closed loop eigenvalues of the LQG system are 
the union of the observer eigenvalues and optimal regulator 
eigenvalues, and hence result In a stable closed loop system, 
the eigenvalues of the controller may lie outside the unit 
circle, therefore causing the controller to be unstable. It 
may therefore be concluded that the LQG control system may 
not be robust. 

There have been three major approaches in alleviating 
the robustness problem that may occur In LQG systems. In 
light of the theorem all three methods will be investigated 
in the same frame work. The first approach is that of Doyle 
and Stein [2]. They developed a robustness recovery 
procedure in which they added fictitious process noise at the 
Plant Input. By controlling the way the fictitious noise 
entered the plant input they recovered the loop transfer 
function (LTR) at the plant input asymptotically as the noise 
Intensity is increased. This method has the drawback that 
the system has to be square. A recent paper by Mad 1 wale and 
Will lams [5] has extended the LTR procedure to minimum phase, 
non— square and 1 eft- 1 nvertab 1 e systems with full or reduced 
order observer based LQG designs. It Is observed that the 
LTR method actually results in the Kalman filter gains being 
forced asymptotically into a region where all eigenvalues of 
the controller lie within the unit circle. The major problem 
in this method is that the Kalman filter is no longer optimal 
with respect to the true disturbances on the plant as its 
eigenvalues have been shifted via the effective adjustment on 


the process noise . Another disadvantage Is that the 40 
dB/decade roll-off associated with the LQG design Is pushed 
out Into the high frequency range where unmodelled high 
frequency modes might be excited and cause Instability. 

The second approach which was Initiated by Gupta [6], 
and by Moore et al [7] In separate papers was to achieve 
robustness in frequency bands where the problems occurred 
without changing the closed- loop characteristics outside 
those frequency bands. Gupta used frequency- shaped cost 
functionals to achieve robustness by reducing filter gain 
outside the model bandwidth. On the other hand Moore et al 
[7] essentially Improvised on Doyle and Stein's LTR method by 
adding fictitious colored noise instead of white noise to the 
process input, thereby relocating both the Kalman filter 
eigenvalues and the controller eigenvalues. Recently 
Anderson et al [8] have Investigated the relations between 
frequency dependent control and state weighting in LQG 
problems. Both of these procedures result In controller 
eigenvalues that lie within the unit circle, thereby 
overcoming the problems stated In the theorem. 

The last approach is due to Okada et al [9]. Their 


approach 

1 s 

drast i ca 1 l y 

different 

from the 

previous 

approaches . 

They 

have changed the structure of 

the 

LQG 

control 1 er 

by 

1 ntroduc 1 ng 

a feed-forward path 

from 

the 

control 1 er 

1 nput 

to the 

control ler 

output . 

This 

Is 

equl val ent 

to 

Introducing an additional 

feedback 

loop 

from 

the output 

to 

the 

1 nput of 

the plant. 

The criteria for 

the 

selection 

of 

the 

gains 

in this path 1 s to 

force 

the 


controller to satisfy the circle criterion. This additional 
loop results In a robust controller with poor response 
properties. Therefore the response Is Improved by 
synthesizing an extended perfect mode 1-fol lowing (EMPF) 
system [9], This approach has the disadvantage that Its 
statistical properties haven't been established. Furthermore 
it Is not always applicable theoretically. However, In 
practice it outperforms Doyle and Stein's LTR method with 
some approximations as described in [9]. 

The approach taken In this paper 1 s an extens 1 on of the 
LTR procedure. A dynamic compensator Is proposed to replace 
the optimal feedback gains so as to recover the open loop 
transfer function at the plant Input. 

II. DERIVATION OF THE DYNAMIC COMPENSATOR 

The LQ optimal controller can be designed for a system as 
described by equations (1) provided that all states are 
available for measurement. The resulting steady state 
controller which Is depicted in Figure 1 will have excellent 
properties as mentioned previously. 



FIGURE 1. The LQ Based Optimal Controller 


The optimal control for this system Is described by the 
f o 1 1 ow i ng equat 1 on . 

u(k) = -K c x(k) (2) 
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In the case that the state measurements are corrupted 

by white gauss Ian noise an LQG controller can be designed In 

which the Kalman filter is used to estimate the states. The 

LQG design results in the following controller equations for 

the Infinite horizon problem. 

The Kalman filter Is described by 

x(k+l ) = Ax(k)+Bu(k)+K f _[y(k+1 )-CAx(k)-CBu(k) ] (3) 

and the optimal control Is described by 

u(k) = -K x(k) (4) 

c 

Figure 2 depicts the LQG system. 



FIGURE 2. The LQG Based Optimal Control 1 er /Observer 

The following three properties of the system have been 
established : 

Pis The closed loop transfer function matrices from r(k) to 
x(k) are Identical In both the LQG and LQ systems. 

P2 : The loop transfer function matrices with the loops broken 
at XX are identical in both implementations. 

P3: The loop transfer function matrices with the loops broken 
at X are generally different. Furthermore the LQG open- loop 
system might possibly have unstable poles. 
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The return difference ratios of the LQG and LQ systems are 
given by the following expressions. 

T . ( z ) = zK [zl-(I-K^C)A+(I-K,-C)BK ] -1 K^C(zI-A)" 1 B (5) 

l qg c r r c r 

T. (z) = K (zl-A) -1 B (6) 

1 q c 

Now define 

A(Z > - T lqg (Z) - T 1q (z > (7) 

It Is now proposed to replace the constant optimal feedback 

gains K c by a dynamic system Viz) In the LQG system and solve 

for It as Mz) approaches zero polntwlse In z. 

A(z) - T , ( z ) | - T. (z) = 0 (8) 

,q9 K .f(z) lq 
c 

A(z) = z'F(z)tzl-(I-K f C)A+(I-K f C)B'l'(z)]" 1 K f C(zI-A)“ 1 B 

- K (zI-A) _1 B = 0 (9) 

c 

{ zV (z)[zl-(l -K..C ) A+ ( I -K^C ) BH* ( z ) ] ~ 1 K^.C-K }(zI-A) -, B = 0 (10) 

f r r c 

Since (zI-A) *B / 0 equation (9) becomes 

zV(z)[zI-(I-K fr C)A+(I-K f C)BH'(z)]“ 1 K f C-K c = 0 (11) 

To solve for ¥(z) It Is necessary to assume that det(K^.C)^0. 
This Implies that the number of outputs should be equal to 
the number of states l.e. m = n. Equation (10) then becomes 

{zV(z)[zI-(I-K f C)A+(I-K f C)BY(z)] -1 -K c (K f C)" 1 }K f C =0 (12) 

or 

z4'(z)[zI-(l-K,C)A+(I-K < -C)BV(z)]“ 1 - K (K^C) -1 = 0 (13) 

tv c r 

(z4'(z)-K c (K f C) _1 [zI-(l-K f C)A+(l-K f C)BM'(z)]} * 

tzI-(l-K f C)A+(I-K f C)B4'(z)] _l = 0 (14) 
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(15) 


(2H'(z)-K c (K f C) 1 [zI-(I-K f C)A]-K c (K f C) 1 ( l-K f C)BV(z) ) * 
[zI-(I-K f C)A+(l-K f C)BM'(z)]~ l = 0 

A(z) = {[zl-K c (K f C)“ 1 (l-K f C)B]4»(z) 

- K c (K f C)" 1 [zI-(I-K fr C)A]} * 

{(zI-( I-K f C)A+( I-^OBVU) ] _1 K f C) * 

(zI-A) -1 b =0 (16) 

Therefore, ff 

T(z) = [zI-K c (K f C) _1 (l-K f C)B]~ 1 K c (K f C)" I [zI-(I-K f C)A] (17) 

Then A(z) = 0. 

III. OBSERVATIONS 

Before an example can be presented to demonstrate the 
effect of the dynamic compensator the following observations 
must be stated. Several problems are encountered In the 
design of the dynamic compensator. The major problem is the 
dependence of the compensator coefficients on the Kalman 
filter gains. Many of the problematic systems that were 
Investigated, l.e. those with unstable controllers, result In 
extremely high compensator gains, and large, hence unstable, 
compensator poles. The reason for this behavior is observed 
to be the high condition numbers associated with K^_ and K^-C. 
Because of this high condition number the matrix (K^C) * has 
extremely large entries, which in turn result in large poles 
and compensator gains. 

A system similar to the one Investigated by Doyle and 
Stein [2], chosen specifically to Illustrate the unstable 
controller poles, resulted in extremely high compensator 
gains, and large unstable poles. Although the compensator 
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recovered the stability margins at the plant Input of the LQG 
system It Is not an acceptable compensator. In an attempt to 
find a physically realizable compensator several systems have 
been tested. Those that result In a realizable compensator 
have the properties that, the matrices mentioned previously 
have low condition numbers, and the controller eigenvalues 
are all within the unit circle. Since the controller Is 
stable the low phase and gain margins associated with the 
problematic LQG systems are not observed, and the dynamic 
compensator does not have a pronounced effect to validate Its 
use In practical systems. 

IV. AN EXAMPLE 

To Illustrate the effects of the dynamic compensator on 
the stability margins of the open loop frequency response the 
following example was considered. 

Let the plant be described by the following state equation : 


x ( k+ 1 ) 


' 1.0 

0.005 ‘ 


1 .25E-5 



x(k) + 

u(k) 

.-0.015 

0.98 


0.005 


0. 18 1 


- 0.3 


w(k) 


( 18 ) 



2.0 

1.0 ' 


1 .0 

0.0 

y (k) = 

a 

O 

• 

o 

0.3648 . 

x(k) + 

i 

o 

• 

o 

1.0 . 


With £{w(k)}=E{v(k)}=0 ; E{w( 1 )w( J ) }=E{ v( 1 ) v( J ) }=2006 f j 
The controller Is : 

u(k)= - [ 50.0 10.0 ] x(k) 


The state estimates are described by equation (3) 


where the 



Kalman filter gains are given by. 


K 


f 


0.0827901406 -0.13645879 

-0.13430101 0.223924574 . 


The compensator as obtained from equation (17) Is 


( 19 ) 


'f(z) 


(z - 0.85914) 

357.546 

(z - 0.125149) 

(z - 0.73884) 

34.2609 

(z - 0. 125149) 


( 20 ) 


To Investigate the effect of the compensator on the system, 
the open loop frequency responses of the system are 
determined at both of the breakpoints defined previously. In 
Figure 3 the Nyquist plots of the system with the constant LQ 
gains are depicted. The Nyquist plots of Figure 4 are those 
of the system with the dynamic compensator. As seen, even 
though there Is a slight Increase In the phase margin the 
difference Is not significant. Also the system exhibits an 
unexpected behavior at high frequencies which decreases the 
gain margin. 

To observe the effect of the compensator on system 
robustness the plant was perturbed to be 


x ( k+ 1 ) 


1.0 

- 0.2 


0.1 


0.9 


x(k) + 


1 .25E-5 
. 0.005 


u(k) 


’ 0.18 
. -0.3 


w(k) (21) 


o 

• 

CVJ 

1.0 


1.0 

o 

• 

o 



x(k) + 



L o.o 

0.3648 


o 

• 

o 

1.0 J 


427 


The Nyqulst plots of Figures 5 and 6 as obtained for 
the open loop responses of the system with and without the 
dynamic compensator Indicate that the effect Is not 
significant, but that there Is definitely an Improvement. As 
seen from Figure 6 there Is an Improvement In both the gain 
and phase margins, at the plant input, l.e., the loop breaking 
point XX. However at point X there Is a decrease In the gain 
margin while a slight increase in the phase margin was noted. 

The following example demonstrates the fact that 
although the compensator designed for the system is not 
practically acceptable It recovers the stability margins at 
the plant input of the LQG system. The plant is the same as 
the one given in (18) with the C matrix chosen to result in 
an unstable controller. The plant output is decribed by the 
following equation : 



2.0 

1.0 ‘ 


' 1.0 

0.0 ' 

y (k) = 



x(k) + 




. 0.0 

0 . 1 


. 0.0 

1.0 


v(k) 

The Kalman filter gains for this system are given by, 
0.143435809 -0.0624081382 

-0.23095458 0.10172081836 

The compensator is described by, 

(z - 0.99095002) 


K f " 


y(z) = 


- 181 195.7489 


(z + 565.52065) 

(z - 0.98813697) 


-1 12641 .206 


( 22 ) 


(23) 


( 20 ) 


(z + 565.52065) 

The Nyquist plots of the system, with and without the 


compensator are depicted in Figures 7 and 8. As seen from 


Figure 8 the LQ open- loop frequency response Is recovered at 
the plant Input, l.e. at point X, when the compensator Is 
used. However the extremely large gains of the compensator 
drastically change the frequency response of the system with 
the loop opened within the control 1 er, at point XX. 

V. CONCLUSION 

As seen from the results described above, the dynamic 
compensator that was designed to mimic the return difference 
of the LQ system at the plant Input of the LQG system did 
result In the anticipated improvement In the stability 
margins at the plant Input. An appreciable Improvement Is 
observed for the LQG system with the unstable controller, 
though there Is no longer any guaranteed stability margins at 
the loop opening point within the control ler, 1 .e. at point 
XX. The same magnitude of Improvement is not seen for systems 
with stable controllers. However, an Increase in the phase 
margins Is observed when the plant model is perturbed. 
Further research may be directed towards Investigating why a 
realizable compensator can not be obtained for all systems 
which have unstable control 1 ers , and hence low stability 
margins, especially for systems that do not have the same 
number of states and outputs. Also an Investigation of the 
effects of the compensator on the time response of the system 
must be performed. 
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